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Corrigendum 


p.  2,  Line  15  "moribind"  should  be  "moribund", 
p.  3,  line  11,  "in"  should  be  "in-". 

p.  5,  line  6,  "basic  process"  should  be  "basic  processes", 
p.  5,  line  11,  "exists"  should  be  "exits", 
p.  6,  line  5  from  bottom,  "(0,“)"  should  be  "  {0,1,2,,..}". 
p.  9,  line  12  from  bottom,  "Remembering"  should  be  "Remember", 
p.  9,  line  3  from  bottom,  "out"  should  be  omitted, 
p.  10,  line  2,  "errormous"  should  be  "enormous". 

p.  11,  line  10,  insert  "in  the  steady  state"  after  "For  fixed  n,"  and 

change  "i.i.d.  random  variables  to  independent 
random  variables". 

p.  17,  line  6  from  bottom,  "family"  should  be  "familiar". 

p.  18,  line  1  and  2,  change  the  sentence  "The  busy  period  is  in  general..."  to 

"The  busy  period  sequence  is  also  a  sequence  of  i.i.d. 
random  variables"  and  then  delete  the  next  sentence, 

"For  the  special  case..." 

p.  18,  lino  15,  insert  "an"  between  "to"  and  "empty". 

p.  24,  line  9,  "sume"  should  be  "sum". 

p.  28,  line  24,  "research"  should  be  "researched". 

p.  31,  Syski  reference  should  be  "Congestion"  not  "Conjection" . 
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ABSTRACT.  The  study  of  queueing  networks  Is  a  rather  recent  addition 
to  queueing  theory.  In  this  paper  we  will  review  about  the  last  20 
years  of  developments  In  the  area.  Primary  emphasis  Is  placed  on 
Jackson  networks  and  the  major  contribution  of  Kelly.  We  discuss 
queue  length  processes,  waiting  time  processes,  busy  period  processes 
and  departure  processes.  We  will  also  note  a  few  new  studies  that 
have  appeared  In  the  area  of  network  flows.  Topics  that  are  not 
discussed  In  detail  In  the  paper  are  briefly  noted  in  the  final  sec¬ 
tion.  The  bibliography  can  serve  as  an  Introduction  for  further 
reading  In  the  area. 


1.  Introduction  and  Some  Background 

1.0  Introduction.  In  this  paper  we  will  briefly  review  some  developments  that 
have  occurred  over  the  last  20  years,  in  an  area  of  applied  mathematics  called 
queueing  network  theory.  The  need  for  such  applied  work  has  been  clear  almost 
since  the  beginning  of  what  is  called  queueing  theory.  Some  of  the  work  of 
A.  K.  Erlang  (see  Brockmeyer,  et  aL  [1948])  seems  to  be  pointing  toward  a  study 
of  Interconnected  systems  of  service  centers.  Certainly  by  the  1930's  It  was 
clear  that  such  work  was  evolving  (see  chapters  7-10  in  Syskl  [I960]).  In  most 
of  these  early  studies,  the  developments  appear  to  be  closely  tied  to  attempts 
to  solve  problems  that  occur  principally  in  telephone  switching  systems. 

Much  of  classical  queueing  theory  (say  up  to  about  1957  or  1960)  was  con¬ 
cerned  with  properties  of  Markov  processes  and  except  for  names  given  to  the 
parameters  and  processes  It  Is  difficult  to  separate  out  the  peculiarities  of 
queues  from  other  Markov  processes  that  were  being  used  as  models,  for  example. 
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In  biological  modelling.  In  fact,  many  of  the  birth-death  models  of  biological 
studies  have  interesting  and  useful  counterparts  in  queueing  theory.  For  this 
reason  a  carefully  drawn  history  of  queueing  theory  would  be  strained  because 
of  che  interrelations  with  other  fields.  The  useful  thing  to  remember  is  that 
in  its  beginning  phases  the  field  was  closely  allied  with  attempts  to  solve 
real  life  problems. 

The  1950's  were  perhaps  the  "golden  age"  of  developments  in  the  field. 
Under  many  impacts  (including  the  introduction  of  more  sophisticated  and  real¬ 
istic  models  into  problems  that  here  to  fore  had  been  the  domain  of  the  indus¬ 
trial  engineer)  the  field  began  Co  take  on  a  life  of  its  own.  In  much  of  the 
1950-60  work  and  extending  even  to  today,  the  work  in  the  field  veered  more  and 
more  away  from  its  applied  bent.  It  was  at  this  time  that  queueing  theory 
started  to  be  sufficiently  arcane  that  many  researchers  were  turned  away  from 
it.  Easy  queueing  problems  had  been  solved.  Many  papers  were  and  had  been 
published  on  special  cases  of  birth-death  processes.  The  field  was  moribind 
after  about  1965-70. 

Starting  about  1965,  the  field  of  computer  systems  analysis  began  to  delve 
into  properties  of  time  sharing  and  interconnected  systems.  Here  the  research¬ 
ers  encountered  many  of  the  same  types  of  problems  encountered  earlier  in  tele¬ 
phone  systems  and  production  systems.  It  was  natural,  therefore,  to  use  exist¬ 
ing,  known  results.  By  1970  and  certainly  by  1973  research  into  corat uters  and 
computer  systems  had  uncovered  a  large  number  of  problems  beyond  Che  work  in 
earlier  queueing  theory. 

Ac  the  same  time  these  developments  in  computer  systems  were  taking  place, 
new  problems  were  occurring  and  old  unsolved  problems  were  becoming  more  pres¬ 
sing  in  telecommunication  theory  and  production  theory.  New  areas  of  applica¬ 
tion  were  evolving  in  military  coamand  and  control  modelling,  disease  modell¬ 
ing,  military  tactics  modelling,  public  sector  models  for  police  fire  depart¬ 
ment  and  medical  emergency  systems  design  and  many  others.  Ail  of  these  topics 
put  demands  on  the  queueing  theory  (and  Indeed  other  areas  of  probability  and 
random  processes)  that  theory  was  unable  to  handle.  Systems  (e.g.  networks)  of 
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queues  became  areas  of  Important  study  In  many  of  the  above  application  rather 
than  single  queues.  Harkov  process  theory  did  not  provide  a  rich  enough  body 
of  knowledge  and  new  ideas  evolved. 

Largely  because  the  applied  topics  put  demands  on  existing  theory  that 
could  not  be  accommodated  quickly  enough,  much  of  applied  work  turned  to  Monte 
Carlo  simulation  of  systems.  But  that  placed  a  new  load  on  the  theory  of  Sta¬ 
tistics  as  well  as  probability,  random  processes  and  queueing.  Because  of  the 
relative  quiescence  of  the  1957-70  period  queueing  theory  has  lost  ground  to 
applied  topics. 


1.1  Purpose.  The  purpose  of  this  paper  is  to  briefly  summarize  where  we  stand 
in  the  study  of  queueing  network  theory.  Time  and  space  do  not  permit  any  in 
depth  discussion  here.  The  bibliography  should  be  consulted  by  the  reader  in¬ 
terested  in  "reading  themselves  into"  the  area.  However,  that  reader  should  be 
forewarned  that  papers  on  these  topics  are  spread  over  a  large  number  of  jour¬ 
nals  of  the  world.  There  is  no  one  best  place  to  look  for  research  papers  or 
to  expect  to  follow  diligently  and  thereby  stay  up  with  the  field.  For  back¬ 
ground  into  queueing  theory  the  reader  might  consult  Syski  [1960]  who  does  a 
masterful  job  summarizing  work  in  queueing  theory  up  to  about  1960.  Klelnrock's 
two  volumes  [1975;  1976]  summarizes  many  known  results  in  queueing  theory  and 
gives  a  nice  discussion  of  many  of  the  problems  of  queueing  phenomena  occurring 
in  computer  systems.  There  are  no  textbooks  on  the  subject  of  queueing  network 
theory  chough  one  can  find  some  discussions  concerning  section  2,  to  follow, 
in  most  post  1975  texts.  Material  in  section  4  is  nicely  presented  but  unfor¬ 
tunately  it  la  in  unpublished  lecture  notes  by  Prof.  Frank  Kelly.  Material  in 
section  5  coupled  with  the  Disney  [1975]  reference  therein  is  a  fair  review  of 
chose  topics.  Topics  in  section  6  have  not  been  pulled  together  in  any  one 


place  to  the  best  of  our  knowledge  so  the  reader  is  on  his  own  there.  Refer¬ 
ences  are  provided  herein  but  chat  list  is  far  from  complete. 

1.2  Some  Background,  Notation  and  Symbolism.  In  order  to  embed  our  future 
discussion  in  the  more  classic  field  of  queueing  theory,  we  will  present  a 
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brief  review  of  a  few  Ideas  In  basic  queueing  theory. 

At  an  Incultlve  level  queueing  theory  la  concerned  with  problems  arising 
In  welting  lines.  One  supposes  that  there  la  something  that  can  perform  a  need¬ 
ed  service,  a  server.  In  the  simpler  cases  It  la  assumed  chat  the  sequence  of 
service  dates  la  a  sequence  of  mutually  Independent  random  variables  that  are 
Identically  and  non- negatively  distributed.  In  queueing  theory  such  a  service 
process  is  called  a  G-type  service  process.  In  the  theory  of  stochastic  proc¬ 
esses  such  a  process  is  called  a  renewal  process. 

The  demand  for  this  service  Is  usually  modelled  with  the  times  between  de¬ 
mands  being  the  variables  of  interest.  These  times  are  assumed  to  be  a  sequence 
of  Independent,  Identically  distributed  random  variables  that  are  non-negative 
(another  renewal  process).  Such  an  arrival  process  Is  called  a  Gl-type  arrival 
process.  It  Is  usual  to  assume  that  the  arrival  process  and  service  time  pro¬ 
cess  are  Independent  processes.  Such  a  queueing  system  is  called  a  Gl/C/- 
queue. 

Queueing  or  waiting  occurs  in  such  systems  whenever  an  arrival  occurs  to 
find  the  server  already  engaged  serving  a  previous  arrival.  If  an  arrival  oc¬ 
curs  and  the  server  is  not  so  engaged,  that  arrival  lmsediately  enters  service, 
under  che  usual  assumptions,  (There  are  studies  that  do  not  Include  this  as¬ 
sumption.  They  are  called  queueing  with  set  up). 

When  service  Is  completed,  the  next  unit  in  the  queue  to  be  serviced  Is 
chosen  and  under  the  usual  assumptions,  that  unit  Immediately  goes  Into  service. 
The  rule  used  to  choose  the  next  unit  to  serve  from  among  all  those  in  the 
queue  Is  called  the  queue  discipline.  The  most  cossnon  assumption  in  queueing 
theory  is  that  the  first  unit  in  the  waiting  line  is  the  first  customer  to  be 
served.  Such  a  discipline  la  called  a  first  ln-first  out  discipline.  For  other 
applications  other  disciplines  have  been  considered.  (In  military  studies  the 
next  unit  to  be  served  may  be  the  one  poalng  the  most  immediate  threat.  In  a 
computer  center  jobs  may  be  assigned  to  classes  of  importance  -  called  priority 
classes  -  and  the  next  job  chosen  is  from  among  those  with  the  highest  priority. 
In  production  systems  the  next  job  to  manufacture  may  be  the  one  whose  promised 
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delivery  dace  -  che  so  called  "due  dace"  -  is  closest.) 

There  are  four  basic  processes  created  by  Che  interaction  of  service  times, 
lncerarrival  times  and  queue  discipline  that  are  of  particular  importance  though 
others  have  been  studied.  The  length  of  the  waiting  line,  called  Che  queue 

length  process  (N(t):  t  ^  0},  is  one  of  the  basic  processes  of  concern.  The 

length  of  time  from  entry  to  exit  of  the  queueing  system  (Qn:  n  »  1,2,---}  call¬ 
ed  che  sojourn  time  process,  is  another  of  the  basic  process.  The  time  from 
first  entrance  of  a  unit  to  an  idle  server  until  first  entrance  of  che  next 
unit  to  an  idle  server  {B  :  n  •  1,2»***}  is  called  the  busy  cycle.  The  time 

between  consecutive  exists  from  che  server  {X  :  n  •  1,2,  —  )  is  called  the 

n 

departure  process. 

Somewhat  more  formally  we  can  define  the  problems  as  follows: 

(1)  Let  0  <_  Tg  <  T  <  Tj-'-  be  a  sequence  of  real  random  variables  representing 

che  times  at  which  an  arrival  occurs  to  a  queueing  system.  Let  A^ 

■  T  -T  ,,  n  -  1,2,  *  •  * .  Assume  A  -  {A  )  is  a  sequence  of  i.i.d.  random 

n  n-1  u  n 

variables  (a  renewal  process).  Call  <A  the  arrival  process  to  the  queue. 

(2)  Let  S  be  a  non-negative  real  random  variable.  Let  S  *  {S  :  n  -  1,2," ••) 

q  n 

be  a  sequence  of  i.i.d.  random  variables.  Call  S  the  service  time  process 
for  the  queue.  We  assume  A  and  S  are  Independent  processes. 

\  'V/ 

(3)  Let  lr„  . (T  )  be  an  indicator  random  variable  taking  values  0  (or  1)  if 

lu.tl  n 

T  does  not  (or  does)  occur  in  (0, t ] . 
n 

(4)  Define  N.(t)  -  I  lrn  ,(T  ).  (N,(t):  t  >  0)  is  called  the  arrival 

A  n»0  c  J  n  A 

counting  process. 

The  four  processes  of  basic  concern  to  queueing  theory  can  be  defined 
formally  in  terms  of  the  A  and  S  process.  Prabhu  [1965]  provides  a  formal 

'Xz 

definition  of  che  queue  length  process.  We  will  not  reproduce  chat  here  but 
rather  we  assume  that  everyone  has  an  intuitive  understanding  of  such  a  process. 
The  sojourn  time  process  is  closely  related  to  the  waiting  time  process  for 
first  ln-first  out  disciplines  which  csn  bs  formally  stated  as 
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W  -W  .  +  S  .  -  A  ,  if  W  ,+S  ,  -  A  >0, 

n  n-1  n-1  n  n-1  n-1  n 


otherwise. 


Then  "  wn  +  sn  the  sojourn  time  of  the  nC^  arriving  customer.  The 

busy  cycle  process  Is  the  sequence  of  first  passage  times  already  discussed. 

If  we  define  Xn  as  the  time  between  the  n  and  (n-l)St  service  completion  then 
the  departure  process  {X^:  n  »  l,2,«-»}  can  be  more  formally  given  as 

S  ,  if  the  nttl  unit  leaves  someone  in  the 

n 

queue, 

I  +  Sn>  if  the  unit  leaves  an  empty  queue. 

Here  1^  is  the  foreward  recurrence  time  in  the  arrival  process  measured  from  the 

s  t 

cime  of  departure  of  the  (n-1)  unit.  1^,  is  called  the  idle  time  of  the  server. 


1.3  Special  Cases.  The  major  thrust  of  queueing  theory  for  much  of  its  life 
has  been  to  compute  formulas  for  the  above  measures  of  effectiveness  uith 
special  assumptions  on  the  arrival  process,  service  process,  queue  discipline 
or  systems  capacity.  For  future  reference,  we  will  record  some  results  for  one 
special  case  here.  The  following  results  as  well  as  many  of  the  special  cases 
are  discussed  in  most  standard  texts  on  the  subject  (for  example,  see  Kleinrock 
(1975;  1976]). 

If  we  assume  that  N^(t)  is  a  Poisson  process  (with  parameter  1  as  is  usual) 
and  Sn  la  exponentially  distributed  (parameter  u  as  is  usual),  the  service  center 
haa  1  server,  the  queue  discipline  is  first  in-first  out  and  the  system's  capac¬ 
ity  la  unlimited  then  the  system  is  called  an  M/M/1  queueing  system.  A  con¬ 
siderable  amount  Is  known  about  the  above  four  processes  (Kleinrock  [1975]). 

For  example, 

(1)  (N(t)}  la  a  Markov  process  with  state  space  (0,»). 

(2)  Pr(N(t)  -  k]  is  known  (but  we'll  not  reproduce  the  results  here.  Essen¬ 
tially  the  probability  is  given  by  an  infinite  sum  of  Bessel  functions  of 
type  -  I  l.e.  Bessel  functions  of  the  second  kind  with  imaginary  argu¬ 


ments  ) . 
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(3)  lim  Pr[N(t)  -  k]  »  (l-p)c'C,  k  •  0,1,2,  •••  for  p  -  A/p  (called  Che  traffic 
C-r» 

lntenaicy).  Such  a  limiting  distribution  exists  if  and  only  if  p  <  1, 
otherwise  the  limit  is  identically  0  for  all  finite  k.  These  limiting 
probabilities,  when  they  exist,  are  called  the  steady  state  probabilities. 
(A)  { Wq >  is  a  random  walk  on  the  non-negative  reals  with  a  delaying  barrier  at 

0.  (see  Feller  (1963]  or  Borovkov  [1976]). 

(5)  prtwn  i.  tl  known  in  principle  (through  the  theory  of  random  walks). 

(6)  lim  Pr(Q  <  t]  ■  1-e  t  >  0.  This  limit  exists  if  and  only  if 

n-*—  n 

0  <  1,  otherwise  the  limit  is  0  for  all  finite  t  >  0. 

(7)  The  busy  cycle  process  is  a  sequence  of  l.i.d.  random  variables  whose 

distribution  is  known  (see  Cox  and  Smith  [1961],  for  example). 

(8)  The  departure  process  {Xn:  n  »  0,1,2, — >  is,  in  principle,  known  for  all 

n  from  (2). 

(9)  In  the  steady  state  (i.e.  t  ♦  “) ,  when  p  <  1,  {X  )  is  a  sequence  of  i.i.d. 

n 

random  variables  that  are  exponentially  distributed  with  parameter  A. 

That  is,  the  departure  counting  process  is  a  Poisson  process  with  the  same 
parameter  as  the  arrival  process,  (see  Burke  [1956]  or  Disney,  ec  al. 
(19731). 

Results  (1),  (3),  (9)  are  useful  to  our  future  discussion. 

2.  Jackson  Networks 

2.0  Introduction.  It  was  recognized  early  in  queueing  theory  chat  a  theory  of 
single  server  systems  was  not  adequate.  Indeed,  at  least  by  the  1930' s  (and 
before  if  one  is  not  too  picky)  research  in  queueing  had  begun  to  explore  sys¬ 
tems  of  queues  or  what  we  shall  call  networks  of  queue.  Erlang,  about  1910  had 
studied  systems  where  many  identical  servers  all  handled  the  same  arrival  pro¬ 
cess  (the  M/M/c  model)  in  telephone  systems.  A  good  discussion  of  these  and 
multiple  server  models  is  given  in  Syski  (I960]  especially  chapters  5  and  6  as 
well  as  chapter  7  to  10. 

2.1  Jackson  Networks.  A  general  model  in  this  historic  mold  that  has  served 
almost  as  the  definition  of  a  queueing  network  was  studied  in  a  paper  by  J.  R. 
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Jackson  (1957].  These  models  and  related  problems  have  come  to  be  called 
"Juckaon  networks."  His  1957  model  assumed: 


(1)  There  ere  1  <  M  <  *  service  centers  (Jackson  did  allow  servicing  center  j 
to  be  comprises  of  more  than  1  server.  However,  to  keep  the  problem  simple 
we  assume  throughout  that  each  servicing  center  has  only  one  server). 

(2)  These  M  servers  are  arbitrarily  connected  by  arcs  over  which  units  travel 
Instantaneously  fast. 


(3)  To  establish  how  units  proceed  through  the  network  one  defines  p  to  be 

the  probability  that  a  unit  completing  service  at  service  center  i  proceeds 


to  service  enter  1  for  its  next  service.  Then  for  each  i,  1  -  Z  p.  .  is 
J  j-1 

the  probability  that  a  unit  exiting  node  i  leaves  the  system.  The  matrix 


P  whose  elements  are  p  is  called  the  switching  matrix  for  the  network. 

<x,  l  j 

Note  chat  these  assumptions  about  switching  imply  that  a  unit  leaving 


service  center  1  chooses  the  next  service  center  to  visit  without  regard 
to  any  ocher  condition.  We  call  this  switching  behavior  a  Bernoulli 
switch. 

(4)  Service  times,  S  n  *  1,2,***  at  center  i  are  i.i.d.  random  variables. 

ni 

These  times  are  each  exponentially  distributed  random  variables  with 

,  ,  M 

parameter  y . .  The  sequences  (S  .}  are  independent  sequences, 
i  ni  iml 

(5)  There  may  be  many  arrival  processes  but  each  one  is  a  Poisson  process  (with 
parameter  if  the  arrival  process  is  to  service  center  i).  These  arrival 
processes  are  Independent  of  each  other  and  of  the  network. 

(6)  All  queue  disciplines  are  first  in-first  out. 

(7)  All  queue  capacities  are  unlimited. 


2.2  The  Queue  Length  Process.  The  results  that  Jackson  obtained  were  rather 

surprising.  Jackson's  major  results  were  as  follows: 

Define  N(t)  to  be  a  vector  whose  jth  element,  N.(t),  j  •  1,2,**',M,  is  the 
'V  J 

queue  length  at  the  jC^  service  center  at  t. 

Then 

(1)  {N ( t ) :  t  _>  0)  is  a  vector  valued  Markov  process. 
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(2)  11a  Pr [N^ (t )  -  k^  N2(c)  -  kj-.-f^d)  - 

-  11a  Pr(N^(t)  -  k^]Pr[Nj(t)  -  kj]  •  •  •  PrlN^U)  -  k^J. 

That  Is,  the  queue  lengch  processes  In  these  Jackson  networks  are  asymp¬ 
totically  (t-~»),  mutually  independent. 

k< 

(3)  lim  Pr(N.  (t)  -  k,]  -  (1-tOb,  j  -  1,2, If  and  only  If  b.  <  1.  Other- 

t-w  3  3  13  3 

wise  this  limit  is  0.  Here  “  a j/Mj  and  a  j  satisfies  the  so  called 
traffic  equation 


a  »  1  +  Pa. 

v  *v  'vv 

a  is  the  M-veccor  of  a,,  X  Is  the  M-vector  of  X,,  P  is  the  switching 

^  J  a.  1  ^ 

matrix.  We  assume  throughout  that  the  traffic  equation  has  a  positive 

solution  (or  alternatively  the  maximum  eigenvalue  of  P  is  less  than  1). 

1, 

In  elemental  form,  this  equation  says  simply  that  a^  la  the  rate  of 

arrivals  to  service  system  J.  This  arrival  rate  is  simply  the  sum  of 

exogeneous  arrivals  to  system  J  (X^)  plus  the  sum  of  all  arrivals  to  j 

from  Inside  of  the  system  (Pa).. 

'vv,  j 

It  is  instructive  to  spend  a  moment  looking  at  what  Jackson  said  and  did 
not  say.  Many  papers  have  been  written  on  Jackson  networks  that  misinterpret 
and  misuse  his  results.  The  major  confusion  occurs  around  the  third  result. 
Remembering  that  in  the  example  of  section  1.3  we  showed  that  the  queue  length 
process  was  asymptotically  (t-*™)  geometrically  distributed  with  a  parameter 
P  -  X/u  <1.  If  p  >  1  the  limit  Is  zero.  Also  recall  that  X  was  the  parameter 
associated  with  the  Poisson  arrival  process  and  u  was  the  parameter  associated 
with  the  exponential  service  times. 

Now  result  (3)  above  Is  also  asymptotically  geometrically  distributed  with 
a  parameter  a^  that  Id  acting  as  an  arrival  rate  and  a  parameter  acting  (Is) 
as  a  servicing  rate.  This  result  led  Jackson  to  state  "This  theorem  (our 
results  (2)  and  (3)  above)  says,  in  essence,  that  at  least  so  far  as  steady 
states  are  concerned  (out  c->) ,  this  system  with  which  we  ere  concerned  behaves 
as  1£  its  departments  (our  centers)  were  Independent,  elementary  (i.a.  M/M/1) 
systems  So  far,  so  good.  The  as  if.  emphasis  Is  Jackson's  and  as  it 
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stands  la  Innocuous  enough.  Jackson  makes  one  more  statement  about  his  results 
thac  may  have  caused  an  errormous  amount  of  confusion.  He  says  "This  conclusion 
la  far  from  surprising  In  view  of  recent  papers  by  E.  J.  Burke  (that  should  be 
P.  J.  Burke)  and  E.  Reich."  It  appears  from  his  references  that  Jackson  is 
here  alluding  to  the  result  (9)  of  section  1.3  which  Is  sometimes  called 
"Burke's  Theorem". 

Confusion  In  queueing  network  literature  has  occurred  by  caking  the  "as 
If"  of  Jackson  too  seriously.  Researchers  seem  to  have  taken  this  to  mean  that 
the  queue  length  processes  are  Independent  and  are  created  by  M/M/1  subsystems. 
As  a  consequence  there  are  papers  In  the  queueing  literature  that  study  queue¬ 
ing  networks  one  node  at  a  time  (the  independence  assumption)  as  M/M/1  queues 
(the  assumption  of  Burke's  theorem).  There  are  other  papers  that  study  sojourn 
times  in  networks  as  sums  of  independent,  exponential  random  varialbes  (see 
section  1.3,  result  (6)).  Unfortunately,  many  authors  have  shown  that  the  flow 
of  units  within  the  network  are,  except  for  special  network  configurations  such 
as  trees,  not  Poisson  processes  and  in  fact,  are  not  even  renewal  processes. 

Thus  each  service  center  In  Isolation  Is  not  only  not  an  M/M/1  queueing  system, 
tc  Is  not  even  a  queueing  system  with  a  renewal  arrival  process.  That  is,  for  a 
time  there  was  serious  confusion  In  the  literature  as  to  what  as  lf_  really 
meant.  The  problem  Is  fairly  well  understood  now.  Section  5  and  Its  references 
discuss  the  topic  more  thoroughly. 

2.3  Waiting  Times.  Waiting  times  in  Jackson  networks  Is  a  rather  unexplored 
area.  Attention  has  recently  turned  toward  that  area  but  research  here  has 
only  started.  There  are  a  few  special  cases  that  have  been  studied  in  detail 
principally  In  what  are  called  tandem  queues. 

One  can  define  a  tandem,  Jackson  network  as  one  having  the  structure  of 
section  2.1  with  the  added  features'. 

(1)  There  Is  only  one  arrival  process  and  it  occurs  to  the  first  server  (in 

terms  of  the  section  2.1  scheme  X^  ”  A,  Xj  ■  0,  j  i  1) . 

(2)  p^j  -  0  unless  j  »  1  +  1  in  which  case  p^  *  1,  j  «  1,2, •••M.  (see  next 
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page)  Since  che  only  results,  with  which  we  are  familiar,  are  concerned 

vlch  first  in-flrst  out  queue  disciplines,  we  assume  this  discipline 

throughout.  We  need  a  bit  more  symbolism  here.  So  define: 

W  -  the  total  time  unit  □  spends  waiting  In  service  center  1.  Call 

this  che  waiting  time  of  unit  n  in  service  center  1. 

Q  -  the  time  spent  by  unit  n  in  service  center  1.  Call  this  the 

m 

sojourn  time  of  unit  n  in  center  1.  W.+S.-0..1*  1.2....M. 

— 1 -  nl  nl  nl 

n  -  1,2, • • • . 

The  first  result  is  from  Reich  [1937]  which  shows: 

For  fixed  n,  {Q^}  is  a  sequence  of  1.1. d.  random  variables  each  of  which 
is  exponentially  distributed. 

Thus,  if  Qn  Is  the  sojourn  time  in  the  system  for  unit  n,  this  result  says 
simply  that  this  sojourn  time  Is  the  sum  of  Independent,  random  variables 

Qn  *  ^  +  <5u2+-”*W 

The  Q  may  depend  on  u, .  Nonetheless  the  distribution  of  Q  can  be  obtained 
i  n 

from  simple  convolution  operations. 

The  next  result  la  surprising.  It  comes  from  Burke  [1964].  For  the 
tandem  queueing  system  he  shows: 

The  sequence  {W^ )  for  n  fixed  and  1  -  1,2, •••M  Is  a  sequence  of  dependent 
random  variables. 

That  Is  sojourn  times  at  successive  service  centers  in  the  network  are 

Independent  but  waiting  times  are  not.  It  is  curious  that  the  sequence 

(W  1  •  1,2,  "'M)  for  fixed  n  Is  a  sequence  of  dependent  random  variables, 
ni 

But  if  to  each  term  In  this  sequence  ve  add  a  random  variable.  Independent  of 
(Wni)  and  exponentially  distributed  (the  to  obtain  a  new  sequence 

{Qn^  *  WQi  +•  Sq^:  1  ■  1,2,-")  that  sequence  is  a  sequence  of  independent 
random  variables  for  each  n. 

These  waiting  time  problems  exhibit  another  nasty  property  that  has  been 
proven  in  two  distinct  cases.  Burke  [1969]  proved  the  first  result.  To  expose 
It  we  must  assume  that  a  service  center  can  have  sere  than  one  server.  It  Is 
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usual  Co  assume  chat  all  service  times  at  a  given  cencer  are  mucually  independ¬ 
ent  and  all  are  exponentially  distributed  with  parameter  Uj  that  depends  only 
on  the  center  (not  on  the  server  in  the  cencer  -  the  servers  are  “identical"). 
With  this  set  up,  Burke  considers  a  three  center  tandem  queue  with  the  first 
center  having  just  one  server,  the  second  center  having  any  finite  number  of 
servers,  (>1),  and  the  third  cencer  having  just  one  server.  Otherwise,  the 
problem  is  that  of  Reich  discussed  above.  Burke  is  then  able  to  show  that  for 
each  n. 

Q  ,  and  Q  _  are  independent  random  variables, 
nl  nz 

Qn2  end  0^  are  Independent  ramdom  variables. 

Qn^  and  Qq^  are  not  Independent  random  variables. 

This  results  is  surprising.  In  this  netvork  one  does  not  have  the  Reich  mutual 
Independence.  In  fact,  one  does  not  even  have  palrvlse  independence. 

Simon  and  Foley  [1979]  have  found  a  similar  result  to  that  of  Burke  in  a 
different  network.  In  their  three  service  center  network,  there  la  one  server 
at  each  center  .  1^  «  1,  1^  -  0,  J  «  2,3, .  The  new  idea  is  that  the  network 

is  not  a  tandem  queueing  netvork.  Rather  one  has  p^2  •  p,  p13  ■  1-p, 
p^j  -  1,  Pjj  «  0,  J  ■  1,2,3  in  the  structure  of  section  2.2  (see  next  page). 
Then  they  are  able  to  show  that: 

and  Qa2  are  independent  random  variables. 

0^2  and  are  independent  random  variables. 

Qn2  and  are  not  independent  random  variables. 

One  conjectures  chat  what  is  happening  here  is  that  in  both  the  Burke 
problem  and  the  Simon  -  Foley  problem  a  unit  in  queue  2  (in  Chose  examples)  may 
be  by-passed  by  units  in  server  1  that  arrive  to  the  system  after  unit  n.  In 
both  cases  then  the  queue  length  and  hence  the  sojourn  time  at  server  3  will 
depend  on  how  many  such  units  by-pass  unit  n  which  in  turn  depends  on  how  long 
unit  n  was  in  server  1. 

Following  this  line  of  thought,  the  following  conjecture  appears  reason¬ 
able.  If  there  is  only  one  path  connecting  any  two  single  service  centers  in 
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Che  network  Chen  one  can  use  Che  resulc  of  Reich  [1957]  to  determine  the  total 
sojourn  time  of  a  given  unit  through  a  Jackson  neework.  If,  on  the  other  hand, 
there  are  multiple  paths  connecting  single  server  service  centers  then  the 
sojourn  times  of  unit  n  at  the  service  center  at  the  start  of  these  multiple 
paths  and  that  at  the  termination  of  the  multiple  paths  are  dependent.  Further¬ 
more,  if  there  can  be  multiple  servers  at  any  service  center  then  the  sojourn 
times  at  service  centers  feeding  this  multiple  server  service  center  and  chose 
being  fed  by  it  are  dependent  unless  the  multiple  server  center  is  first  or 
last  in  the  sequence. 

Unfortunately,  we  do  not  know  the  nature  of  these  dependencies.  Neither 
do  we  know  Che  sojourn  times  through  either  the  Burke  or  Simon  and  Foley  net¬ 
works.  This  is  an  area  in  need  of  considerably  more  study. 

There  is  yet  one  more  problem  with  sojourn  times  in  these  Jackaon  networks. 
In  perhaps  the  simplest  non-trivlal  Jackson  network  one  takes  all  of  the  assump¬ 
tions  of  section  2.1  with  the  added  assumption  of  single  servers  at  each  service 
center.  The  only  alteration  is  to  have  Just  one  service  station  with  p^  -  p 
(see  next  page).  We  call  this  a  queue  with  instantaneous,  Bernoulli  feedback. 
The  definition  of  sojourn  time  needs  a  modification.  So  let: 

Q*  -  the  sojourn  time  of  unit  n  the  it*'  time  that  unit  passes 
through  che  server.  Then 

Q  -  the  sojourn  time  of  unit  n. 
n 

0  -  Q1  +  Q2+-*--K)k 
n  n  n  n 

where  k  is  a  geometrically  distributed  random  variable. 

It  has  been  shown  by  Takacs  [1963]  and  Disney  and  D'Avlgnon  [1978]  that  for 

each  fixed  n,  (Q1,  1  "  1,2,'**}  is  a  Markov  renewal  process.  Takacs  and  Disney 
n 

and  D'Avlgnon  each  solve  the  problem  for  slightly  more  general  cases  chan 
Jackson  networks  using  rather  different  methods.  Once  again  as  in  che  Burke 
and  Simon  and  Foley  examples  the  lack  of  independence  seems  to  be  coming  from 
what  we  loosely  called  by-passing. 


FIGURE 
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2.4  Sumary  of  Results.  In  sumary,  unlike  the  Jackson  queue  length  process 
results  that  have  been  obtained  and  are  elegant  in  appearance,  waiting  time  and 
sojourn  time  properties  are  largely  unknown.  For  tandem  queues  studied  by 
Reich  the  problem  can  be  considered  solved.  For  general,  single  server, 

Jackson  networks  that  do  not  have  multiple  paths  between  any  two  service  centers 
In  the  network,  it  appears  that  the  Reich  results  can  be  used  to  obtain  results 
easily.  For  more  general  networks  of  Jackson  type,  the  sojourn  time  problem  is 
largely  unsolved.  Because  of  its  Importance  to  many  areas  it  is  one  of  the  most 
pressing  problems  in  queueing  network  theory. 

3.  Busy  Periods  and  Departure  Processes 
3.0  Introduction.  The  queue  length  process  and  sojourn  times  have  been  the 
major  topics  of  Interest  to  queueing  theory  since  its  beginning.  While  of  some 
importance  to  systems  analysis,  the  busy  period  and  to  a  lesser  extent  (until 
rather  recently)  the  departure  process  from  single  service  queues  have  been  of 
less  importance.  The  same  result  is  true  for  Jackson  queueing  networks  as  we 
shall  see  below. 

3.1  The  Busy  Period.  One  can  define  two  related  processes  of  Interest.  One  is 

called  the  busy  period.  The  other  is  cslled  the  busy  cycle.  Both  are  first 

passage  probabilities.  The  busy  period  is  the  time  from  first  entrance  of  a 

unit  to  an  empty  queue  to  first  exit  of  a  unit  that  leaves  the  system  empty. 

The  busy  cycle  is  the  first  return  time  from  entrance  of  a  unit  to  an  empty 

queue  to  the  next  time  of  entrance  to  an  empty  queue.  If  the  queue  is  ergodic 

such  points  occur  infinitely  often.  If  the  queue  is  not  ergodic  such  points, 

except  possibly  the  first  ona,  may  not  exist.  Almost  all  studies  of  the  busy 

cycle  and  busy  period  with  which  are  are  family,  assume  these  entrance  times 

occur  infinitely  often  (see  Cox  and  Smith  [1961]). 

In  single  server  queueing  theory  in  which  {Aq}  is  a  sequence  of  1.1. d. 

random  variables,  (S  }  is  also  a  sequence  of  1.1. d.  random  variables  and  (A.) 

u  n 

and  (S  )  are  Independent  sequences,  the  sequence  of  busy  cycles  (B  :  n«l,2,...) 
n  n 

is  a  sequence  of  1.1. d.  random  variables  if  the  origin  of  the  time  scale  is 


18 


Ralph  L.  Disney 


taken  as  a  point  of  arrival  to  an  empty  queue.  The  busy  period  is  in  general 
not  a  sequence  of  1.1. d.  random  variables  because  successive  periods  are  not 
Independent.  For  the  special  case  In  which  (N^Ct)}  Is  a  Poisson  process,  the 
busy  period  process  as  well  as  the  busy  cycle  process  are  both  sequences  of 
1.1. d.  random  variables.  General  distributional  results  are  known  for  the 
cases  In  which  the  two  processes  are  sequences  of  1.1. d.  random  variables  (for 
example,  see  Kleinrock  [1875]). 

Unfortunately,  the  related  problems  In  queueing  networks  are  In  a  much  less 
well  developed  form.  To  the  best  of  our  knowledge  there  are  no  known  results 
for  busy  periods  or  busy  cycles  for  Jackson  networks.  The  area  could  3tand 
some  study. 

Stating  the  problem  that  needs  to  be  considered  is  rather  easy.  Recalling 

from  section  2.2  (Item  1)  that  (N(t)>  la  a  vector  valued  Markov  process,  the 

% 

busy  cycle  process  Is  then  simply  the  time  from  first  entrance  of  an  arriving 
unit  to  empty  network  to  the  next  time  of  first  entrance  of  an  arriving  unit  to 
an  empty  network.  In  the  busy  period  case  the  problem  is  to  determine  the  time 
from  first  entrance  to  an  empty  network  by  an  arriving  unit  to  the  time  at  which 
a  departing  unit  leaves  behind  an  empty  network.  We  surmize  that  such  points 
occur  infinitely  often  for  ergodlc  networks  but  as  pointed  out  above  we  know  of 
no  results  for  the  busy  period  or  busy  cycle  of  a  Jackson  queueing  network. 

3.2  Departure  Processes.  There  are  two  problems  here  that  lead  to  interesting 
questions  related  to  our  discussion  of  the  queue  length  process  and  waiting  time 
process.  One  problem  is  concerned  with  the  nature  of  the  departure  processes 
from  Che  Jackson  network.  The  other  problem  is  concerned  with  the  nature  of 
the  departure  processes  from  single  centers  in  the  network.  To  simplify  our 
discussion  we  will  assume  the  network  Is  Irreducible  In  the  sense  that  the 
switching  matrix  P  Is  an  Irreducible  matrix.  More  general  cases  have  been 

'V 

studied.  The  Interested  reader  Is  referred  to  Melamed  [1979]  and  its  references 
for  these  other  cases. 

When  Dj  <  1  for  1  •  1,2, — M  every  entering  unit  eventually  leaves  the 
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system.  Then  from  Melamed  [1979]  we  have : 

(1)  In  Jackson  netvorks  with  single  server  service  centere  If  J  Is  a  node  from 
which  departures  from  the  network  occur  then  the  departure  process  from 
the  network  at  this  center  Is  a  Poisson  process  with  parameter  a^. 

(2)  The  collection  of  Poisson  processes  of  departures  from  the  network  are 
mutually  independent. 

The  first  of  these  results  is  quite  in  keeping  with  the  Burke  theorem 
(section  1.3,  item  (9)).  The  second  result  is,  at  first  glance,  rather  surpris¬ 
ing.  One  would  expect  that  the  network  Itself  Imposed  some  dependencies  on  the 
departing  processes. 

When  one  turns  co  consider  departure  processes  from  individual  service 
centers  in  the  network  things  became  a  bit  more  complicated.  For  the  time  being 
we  will  continue  the  discussion  within  the  framework  of  Jackson  networks. 
Furthermore,  we  must  distinguish  two  cases.  If  p^"^  >  0  is  defined  as  the  n 
step  transition  from  1  to  1  In  the  switching  process,  then  there  Is  some  path 
leading  from  service  center  1  back  to  that  service  center.  In  this  case  we 
will  say  that  service  center  i  has  feedback.  Otherwise  we  will  say  service 
center  1  does  not  have  feedback.  The  latter  case  we  can  dispose  of  quickly. 

If  service  center  1  does  not  have  feedback,  the  departure  process  is  a 
Poisson  process  with  parameter  a^. 

The  service  center  with  feedback  requires  distinguishing  two  processes. 

In  one  process  units  leaving  service  center  i  will  eventually  return  to  1.  In 
the  ocher  process,  units  leaving  service  center  1  will  never  return  to  1.  Call 
the  former  process,  the  feedback  stream  and  the  latter  the  departure  stream. 

(see  figure  3).  Then  again  from  Melamed  [1979]  we  have: 

(1)  The  departure  stream  is  a  Poisson  process. 

(2)  The  feedback  stream  is  not  a  Poisson  process  and  in  fact  is  not  a  sequence 
of  i.l.d.  random  variables. 

We  can  flash  out  item  (2)  a  bit  more  in  the  case  >  0,  that  is  feedback 
occurs  in  one  step  -  the  so  called  instantaneous  feedback  case.  In  that  case 
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the  total  process  of  units  leaving  the  service  center  (called  the  output  pro¬ 
cess)  Is  s  Markov  renewal  process  whose  transition  functions  are  known.  Then 
it  has  bssn  shown  by  Disney,  ec  al.  [1979]  that  the  output  process  Is  never  a 
Poisson  process  nor  Is  It  a  renewal  process.  However,  the  departure  process  Is 
a  Poisson  process  (parameter  X).  The  feedback  process  Is  not  a  renewal  process. 
There  la  reason  to  believe  that  the  departure  process  and  the  feedback  process 
are  not  Independent  random  processes  but  we  know  of  no  proof  either  way  here. 

These  Malaaed  and  Disney  et  al.  results  raise  some  Interesting  questions. 

In  these  Jackson  networks  as  was  noted  in  section  2.2  Item  3,  queue  lengths  at 
Individual  service  centers  act  as  if  they  were  independent,  M/M/1  queues.  Yet 
If  the  network  has  feedback  loops,  the  flow  on  those  loops  Is  not  a  Poisson 
process  nor  even  a  sequence  of  1.1. d.  random  variables.  It  Is  this  property 
(l.e.  the  distinction  between  properties  of  the  network  and  properties  of  the 
Individual  service  centers  In  the  networks)  that  seems  to  have  created  confu¬ 
sion  In  some  applications  of  the  Jackson  network  results  both  in  the  studv  of 
the  queue  length  process  and  that  of  the  waiting  time  process. 

4.  Extensions  to  the  Jackson  network  Theory  of  1957 
4.0  Introduction.  Following  his  1957  paper,  Jackson  next  published  a  paper  In 
1963  In  which  he  followed  the  basic  Ideas  of  the  earlier  paper.  The  new  ideas 
were  to  allow  the  arrival  processes  to  the  network  to  be  birth  processes  whose 
parameters  could  depend  on  the  total  number  of  units  in  the  network.  Similarly, 
the  service  time  processes  were  death  processes  with  parameters  depending  on  the 
number  of  units  at  a  given  service  center.  In  this  way  the  queue  length  process 
(N(t))  becomes  a  vector  valued  birth-death  process. 

4.1  Queue  Length  Processes  in  the  Vector-valued  Birth-Death  Process.  We  will 
not  reproduce  the  exact  form  of  Jackson's  1963  results.  They  would  require 
Introducing  a  large  amount  of  new  symbolism  and  currently  available  work  which 
we  shall  discuss  later  Includes  these  results.  However,  it  is  Important  to 
sunnarlze  the  findings  of  Jackson  (at  a  cost  of  Imprecision)  because  they  have 
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lead  many  others  to  explorations  in  queueing  networks  In  an  attempt  to  gener¬ 
alize  the  concept  of  a  Jackson  network. 

In  his  1963  paper,  Jackson  finds  that  the  Joint  probability  of  the  vector 

N(t)  Is  asymptotically  (t-*»)  geometric.  The  first  term  in  this  geometric  die- 
% 

trlbutlon,  representing  the  probability  that  the  entire  network  is  Idle,  how¬ 
ever,  cannot  be  factored  (except  In  special  cases)  Into  a  form  that  would  Imply 
that  the  individual  queue  length  processes  were  Independent. 

Since  these  results  appeared,  considerable  effort  has  been  expended  trying 
to  determine  approximations  and  easy  ways  to  compute  the  Initial  term.  Other 
research  effort  has  been  expended  on  exploring  the  "product  form"  (Implied 
above)  of  the  solution  to  many  networks.  There  are  no  up  to  date  summaries  of 
the  large  amount  of  work.  The  Kleinrock  [ 1975 ; 1976 ]  books  are  basic.  The 
papers  of  Kelly  [1976;  1978]  and  Schassberger  [1977;  1978]  trace  some  of  the 
work. 


6.2  A  Generalization.  In  a  1976  paper,  Kelly  significantly  generalizes  the 
concept  of  Jackson  network  and  provides  Important  extensions  to  the  concept  of 
"product  forms"  of  solutions.  In  an  unpublished  series  of  lecture  notes  (Kelly 
[1978])  he  provides  further  insights  and  significantly  broader  applications  of 
his  study.  We  will  follow  his  1976  publication  because  It  Is  accessible  In  the 
open  literature. 

Suppose  that  there  can  be  1  types  of  units  entering  the  network.  Units 
of  type  lei  enter  the  network  as  a  Poisson  process  with  rate  v(l)  and  pass 
through  the  servers  according  to  the  path 

r(i,l)r(l,2)...r(l,S(l)) 

before  leaving  the  system.  Thus  at  stage  s  (s  ■  1,2, ...S(i))  of  his  route,  the 
unit  Is  at  queue  r(l,s). 

Within  each  queue,  the  units  ars  ordered  so  that  there  ars  units  In  posi¬ 
tions  1,2, ••■Oy  n^  la  the  total  number  of  units  in  quaue  J. 

Each  unit  requires  a  random  amount  of  service.  This  service  time  Is  an 
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exponentially  distributed  random  variable  with  mean  1. 

There  ia  a  single  server  who  supplies  a  total  service  effort  at  rate 
♦jCn^)  in  such  a  way  that  Yj(4,nj)  of  this  effort  Is  directed  to  the  unit  in 
position  4.  When  this  unit  leaves  the  jC^  service  system,  all  units  behind  it 
move  up  a  apace. 

When  a  unit  arrives  at  service  center  j  It  moves  Immediately  into  loca¬ 
tion  4  with  probability  6^(4,  n^+l).  Units  formerly  occupying  spaces  4, 

4  +  l,*’*^  are  moved  to  spaces  4+1,  4  +  2,*,#n^  +  1. 

Such  a  structure  is  rather  general  for  queueing  network  behavior.  The 

queue  discipline  of  the  earlier  Jackson  networks  has  been  considerably  gener¬ 
alized.  Arrival  processes  are  still  Poisson  but  note  the  arrival  rate  may 
depend  on  the  "type"  of  the  unit.  "Type"  may  be  associated  with  the  path  taken 
by  the  arrival  simply  by  associating  an  arrival  "type"  with  the  path  that 
arrival  will  follow.  Service  times  here  are  also  more  general. 

Another  important  aspect  of  the  Kelly  paper  is  its  method  of  determining 
the  limiting  probability  vector  for  the  queue  length  random  process.  Whereas 
the  earlier  Jackson  network  paper  of  1957  and  1963  proceeded  from  the  structure 
of  the  problem  to  set  up  the  usual  limit  form  of  the  Kolmogorov  equations  of 

the  Markov  process  (N(t)},  Kelly  prefers  to  work  with  relations  embodied  in  a 

v 

reversad  process.  It  appears  that  when  such  an  approach  can  be  made  to  work, 
detailed  calculations  are  obviated.  The  Kelly  lecture  notes  expand  on  this 
point  and  considers  these  methods  for  a  wide  variety  of  problems  in  multidimen¬ 
sional  Markov  processes,  Including  the  queueing  networks  of  this  1976  paper. 

Kelly  starts  with  a  stable,  conservative,  regular  Markov  chain  (all  of 
these  networks  discussed  so  far  have  these  properties).  Then  it  is  well  known 
to  the  theory  of  such  processes  that  if  1,  j  are  vector  valued  states  in  these 
networks,  then  a  solution  to  the  equations 

ni»-  o 

with  II  ^  0  and  ■  1  is  unique  and 
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(The  elements  of  IT  are  the  limiting  probabilities  -  the  so  called  "steady  state" 
probabilities  -  of  the  Markov  process). 

Nov  it  Is  known  that  If  (N(t)}  Is  a  Markov  process  In  equilibrium  (e,g. 
che  Initial  vector  for  the  process  is  n)  there  then  exists  another  process  - 
called  the  reversed  process.  (N(-t)},  that  Is  a  Markov  process  in  equilibrium. 
The  reversed  process  has  the  same  limiting  probability  vector,  II  but  its 
inf inices8lmal  generator  may  not  be  Chat  of  (N(t)).  (If  the  two  processes  have 
the  same  inf lnicessioal  generator  then  (N(t)}  Is  said  to  be  reversible).  In 
general  If  q(l,J)  is  the  (l,j)  element  of  the  lnflnltesslmal  generator  of 
(N(t)}  and  q'(i,J)  chat  of  the  lnflnltesslmal  generator  of  (N(-t))  and  if 
q(i) ,  q'(l)  are  Che  corresponding  diagonal  elements  of  the  lnflnltesslmal 
generator  of  {N(t))f  and  (N(-t)}  respectively,  then  we  have 

(4.2.1)  H(l)q(i,j)  -  n(j)q'(j,i) 

and 

q(i)  •  q’(j). 

(If  the  process  Is  reversible  these  equations  are  called  the  equations  of  de¬ 
tailed  balance). 

The  extremely  useful  result  Is  that  for  the  network  set  up  by  Kelly,  one 
can  find  q(l,J)  and  q'(i,j)  rather  easily.  This,  along  with  (4.2.1)  and  the 
uniquess  of  H  as  a  probability  vector  then  allovs  one  to  determine  II. 

For  his  network,  Kelly  defines  a  two-tuple  Cj(l)  ■  ((tj(i),  s^d))  where 
tj(t)  denotes  the  "type"  of  unit  in  position  l  in  queue  J  and  Sj(i),  as  pre¬ 
viously  defined,  represents  the  position  along  its  path  reached  by  this  ith 
unit  in  queue  j.  It  is  shown  that  for  the  vector 

Cj  -  (Cj  (1),  Cj (2)- ••Cj (nj)) 

C  -  (crc2,...cM) 

is  an  Irreducible  Markov  process  on  a  countable  state  space.  The  lnflnltesslmal 
generator  for  this  process  and  Its  reversed  process  Is  found.  Then  using  the 
results  on  reversed  processes  stated  above,  Kelly  shows  that  his  network  has 
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Che  produce  form  of  solution  which  Is  determined  up  to  a  normalizing  constant, 

4.2  Another  Generalization.  Kelly  generalizes  his  model  one  more  step  and  in 
the  process  provides  the  basis  for  talcing  a  major  step  out  of  the  restrictive 
assumption  of  Poisson  processes  and  exponential  service  times.  Unfortunately, 
this  step  has  a  price  to  pay. 

The  first  step  Is  to  generalize  the  service  time  assumption  of  section 

4.2.  Now  Instead  of  service  times  being  exponentially  distributed  random 
variables.  It  Is  assumed  that  when  at  queue  j,  the  unit  that  is  at  stage  s  of 
Its  pach  requires  an  amount  of  service  that  Is  the  sume  of  Z(j,s)  independent. 
Identically  distributed,  exponential  random  variables  (rate  d(j,s)).  That  Is, 
service  clmes  are  now  gamma  distributed  random  variables  with  parameters  Z(j,s) 
and  d(J,a). 

The  other  assumptions  of  section  4.2  are  retained  except  that  It  Is 
required  that 

(4.3.1)  djU.Uj  +  1)  -  v  (t.j  +  1). 

For  this  problem  the  state  space  of  the  Markov  process  of  Interest  must 
now  become  a  three-tuple.  A  state  now  is  defined  by  t^d),  s^U),  as  In  section 

4.2,  and  Xj (Z)  which  denotes  the  phase  of  service  currently  occupied  by  the 
unit.  Then,  on  this  three-tuple  space  one  can  define  a  Markov  process  whose 
states  are  the  three-tuples.  This  process  is  irreducible  and  the  state  space 
la  countable.  The  process  has  a  reversed  process,  Its  inf initeaslmal  generator 
and  that  of  the  reversed  process  can  be  found.  As  before  the  properties  of 
reversing  are  used  and  it  Is  shown  that  the  process  has  a  product  form  of 
solution. 

4.4  A  Major  Generalization.  Were  the  Kelly  results  to  stop  here  they  would 
make  Interesting,  perhaps  useful,  contributions  to  the  theory  of  Jackson  queue¬ 
ing  networks.  The  restriction  of  Poisson  arrival  processes  and  either  expo¬ 
nentially  distributed  or  gama  distributed  service  times  would  preclude  a  wide 
spread  use  of  the  results  (although  much  of  the  computer  systems  analysis 
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literature  finds  these  conditions  to  be  reasonable  for  many  computer  studies). 
But  more  is  available. 

Under  the  conditions  of  the  model  of  section  4.3,  Kelly  conjectures  that 
his  results  go  through  for  G-type  service  times  (see  section  1.2).  The  con¬ 
jecture  is  based  on  a  result  of  Whitt  [1974]  which  shows  that  finite  mixtures 
of  gams  distributions  are  dense  in  the  set  of  arbitrary  non-negative  distribu¬ 
tions.  Though  Kelly  does  not  prove  that  his  model  of  section  4.3  extents  to 
G-type  servers,  it  is  proven  with  the  requisite  care  by  Barbour  [1976], 

Finally,  Kelly  drops  the  Poisson  arrival  process  assumption  that  has  run 
through  his  work.  Commenting  on  his  models  in  our  sections  4.2  and  4.3  he 
notes  that  one  can  allow  these  arrival  processes  to  be  birth  procesaes  whose 
parameter  depends  on  the  total  number  of  units  in  the  system.  Recall  that  this 
step  was  made  in  the  Jackson  [1963]  paper. 

4.5  Comments.  The  Kelly  work  probably  represents  the  state-of-the-art  in  the 
study  of  queueing  networks  originally  arising  out  of  the  papers  of  Jackson. 

Work  continues  on  these  problems  and  the  Kelly  work  will  probably  be  mined  for 
quite  a  while.  If  condition  (4.3.1)  could  be  removed  from  the  Kelly  model  and 
still  obtain  computable  equilibrium  solution,  one  would  have  a  major  contribu¬ 
tion  to  the  theory  of  queueing  netvorka. 

Concerning  waiting  time,  busy  period,  and  busy  cycle  analysis  we  know  of 
no  results  presently  available.  Concerning  departure  processes,  Kelly  presents 
some  results  on  the  departures  from  the  network.  These  processes  arc  independ¬ 
ent,  Poisson  processes  for  the  models  studied  in  our  sections  4.2  and  4.3. 

It  has  been  noted  by  many  authors  (for  example,  sec  Kelly  [1976])  that 
since  the  Jackson  limiting  probability  vector  depends  on  the  assumptions  of  the 
arrival  process  and  service  time  process  only  through  the  expected  values  of 
these  processes,  similar  results  may  wall  hold  for  more  general  arrival  and 
service  tlma  processes.  That  is,  results  such  as  those  obtained  may  be 
insensitive  to  the  distributional  assumptions  of  the  model.  Schaesberger  [1977; 
1978]  explores  this  topic  in  more  detail  and  provides  a  bibliography  for  further 
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5 .  Flow  Processes 

5.0  Introduction.  Many  writers,  in  recent  years,  have  coae  to  realize  that  the 
results  of  the  1957  Jackson  paper  are  remarkable.  Even  though  the  results  imply 
thac  the  queue  length  processes  act  as  i_f  they  are  those  of  Independent  M/M/1 
queues,  they  are.  In  fact,  not.  The  result  haa  lead  to  a  collection  of  papers 
investigating  the  properties  of  the  flow  of  units  vithln  the  network.  Such 
analysis  is  important  if  one  is  to  gain  an  understanding  of  these  flow  proces¬ 
ses,  if  one  la  to  generalize  the  simple  switching  structure  in  Jackson  networks. 
If  one  la  to  gain  an  understanding  of  waiting  times  and  if  one  is  to  gain  some 
understanding  of  sampling  data  produced  by  service  centers  in  the  network. 

The  only  attempt  to  survey  this  overall  field  with  which  we  are  familiar 
is  Disney  [1975].  That  paper  is  somewhat  out  of  date  by  now.  However,  the 
bibliography  therein  la  rather  complete  up  to  July,  1975. 

5.1  A  Few  New  Results.  The  study  of  flow  processes  in  these  networks  consists 
primarily  of  the  study  of  five  subareas  called:  decomposition,  recomposition, 
departures,  feedback  queues  and  queues  with  non-renewal  arrivals.  Few  new 
results  seem  to  have  appeared  since  1975  concerning  the  area  of  decomposition. 
Therefore  we  refer  to  the  Disney  [1975]  paper  for  an  up  to  date  discussion  on 
that  topic.  Recomposition  studies,  likewise,  have  received  scant  attention 
since  1975  as  is  also  true  for  queues  with  non-renewal  arrivals.  We  suggest 
chat  the  Interested  reader  consult  the  above  review  for  work  in  this  area. 

Thera  have  been  two  major  results  on  departure  processes  in  Jackson  net¬ 
works  and  Che  networks  of  Kelly.  In  particular,  Kelly  shows  thac  departures 
from  his  networks  (both  those  discussed  in  section  4.2  and  chose  discussed  in 
section  4.3)  are  Poisson  processes  and  thac  the  several  streams  of  departures 
from  che  service  centers  are  mutually  Independent  processes.  (See  Kelly  [1976]). 
In  Melamed  [1979],  che  results  of  Kelly  for  Jackson  [1957]  netvorks  are  veri¬ 
fied  using  quite  different  method.  In  addition  Melamed  shows  that  flows  on 
arcs  internal  to  Jackson  network  are  Poisson  processes  on  those  arcs  joining 
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non-cosaunicating  classes  of  states  In  the  switching  process.  On  arcs  within 
communicating  classes,  the  flows  are  not  Poisson  processes  and  In  fact  are  not 
even  renewal  processes. 

In  an  attempt  to  study  these  flows  In  subsets  of  communicating  service 
centers,  Disney  and  D'Avlgnon  [1978]  and  Disney,  McNlckle  and  Simon  [1979]  study 
a  simple  case  In  detail.  This  case  is  called  a  queue  with  instantaneous  feed¬ 
back  (see  p.16).  The  1979  paper  studies  four  random  processes  in  these  queues 
to  show  chat  the  departure  process  from  the  systesi  is  a  Poisson  process  If  and 
only  if  service  tines  are  exponentially  distributed  randoo  variables  (see  also 
Disney,  Farrell  and  de  Morals  [1973]  on  this  topic  for  single  server  queueing 
systens.)  In  case  of  renewal  process  service  tines  the  departure  process  Is 
not  only  not  a  Poisson  process,  it  Is  not  a  renewal  process.  T.t  Is  shown  that 
the  process  Is  a  Markov  renewal  process  and  Its  transition  functions  are  given. 
Flows  Internal  to  the  network  are  never  (for  any  service  tine  procese)  Poisson 
processes  (except  for  the  trivial  case  In  which  there  Is  no  feedback) .  In 
general  these  internal  flowe  are  Markov  renewal  processes  and  In  the  single 
case  studied  by  Disney,  McNlckle  and  Simon  the  transition  functions  for  sone 
of  these  Markov  renewal  processes  are  given. 

The  Disney  and  D'Avlgnon  paper  [1978]  Is  a  najor  study  of  queues  with 
Instantaneous  feedback.  In  particular,  the  arrival  process  Is  allowed  to  be  a 
Markov  renewal  process,  service  times  may  depend  on  the  state  of  the  arrival 
process  and  the  switching  probabilities  may  depend  on  queue  length  increments, 
previous  switching  decisions,  types  of  units  being  switched  and  the  amount  of 
service  time  received  by  the  customer  being  svltched.  This  psper  Is  probably 
the  state-of-the-art  for  the  study  of  queues  with  instantaneous  feedback.  It 
Includes  references  to  all  work  with  which  we  were  familiar  in  mid  1978.  The 
details  are  far  too  complex  to  summarize  here.  Us  ask  the  Interested  reader 
to  consult  the  paper. 

Current  research  is  being  conducted  on  queueing  processes  as  wall  as  flow 
processes  In  queues  with  delayed  feedback  (see,  for  example  Foley  [1978]). 

Except  as  noted  In  Disney  and  D'Avlgnon  [1978]  almost  all  of  this  work  is 
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concerned  with  non-Jackson  networks  with  emphasis  on  queue  length  processes  and 
flow  processes.  We  know  of  no  results  other  than  those  already  cited  on  the 
waiting  time  process. 


6 .  Sunaary 

6.0  Stannary.  We  have  attempted  to  revlev  In  a  few  pages,  more  than  20  years 
worth  of  research  In  the  field  of  queueing  network  theory.  To  accomplish  this 
in  such  a  short  space  we  have  concentrated  on  two  topics:  Jackson  networks  and 
flow  In  networks.  Our  primary  emphasis  has  been  on  the  Jackson  network  results. 
In  areas  of  application  these  are  the  results  that  are  of  primary  importance. 
Under  the  pressing  restrictions  of  time  and  space  ve  have  concentrated  only  on 
the  basic  Jackson  work  and  the  important  Kelly  work.  While  we  have  alluded  to 
other  work,  we  have  by  no  means  provided  a  definitive  state-of-the-art  survey. 
Considerable  work  haa  been  done  on  Jackson  networks  In  the  past  15  years.  This 
work  is  to  be  found  principally  In  the  literature  of  the  computer  scientist 
whosa  interests  in  these  topics  seems  to  have  revitalized  that  field.  We  can 
only  hope  that  the  reader  Interested  in  a  host  of  results  and  fascinating 
appllcationa  will  conault  this  literature.  The  best  starting  point  is  probably 
tha  two  volumes  of  Klelnrock  (1975;  1976J  and  especially  the  Interesting 
chapters  4,5,6  of  volume  II  which  present  some  of  the  basic  queueing  problems 
occurring  In  computer  networks  as  well  as  an  Interesting  discussion  of  trials 
and  tribulations  of  applying  known  results  to  the  design  of  a  large  scale 
system.  Beyond  that  we  can  only  suggest  chat  the  Interested  reader  peruse  the 
journals  of  computer  science  (e.g.  J,.A.£.M.  or  Acta  Informatics)  as  these  topics 
continue  to  be  research  and  applied. 

The  study  of  flow  processes  In  queueing  networks  Is  fragmented  at  present. 
There  are  many  results.  We  have  mentioned  a  few.  There  Is  much  that  we  have 
not  said  and  much  that  remains  to  be  said.  We  have  not  discussed  the  Interest¬ 
ing  work  referenced  In  Schassberger  (1977;  1978]  that  Is  attempting  to  tie 
queueing  theory  Into  the  more  general  field  of  stochastic  point  process  theory. 
Indeed,  It  Is  our  view  that  this  link  up  is  natural.  For  the  study  of  flow 
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processes  in  networks,  it  is  natural  to  think  of  the  network  and  its  components 
as  operators  on  random  point  processes.  That  view  will  probably  provide  greater 
generality  and  deeper  insights  into  these  flow  processes  than  is  now  possible. 

At  present,  results  on  the  study  of  flow  processes  in  queueing  networks 
appear  in  print  sporadically.  Unfortunately,  such  results  when  they  do  appear 
are  published  in  literature  through  the  world.  The  journals  Appl.  Prob.  and 
Ad v.  Appl.  Prob.  are  necessary  reading  but  not  sufficient. 

There  are  many  other  topics  in  these  areas  that  we  have  not  even  mentioned. 
The  useful  computational  work  of  Wallace  [1974]  and  Neuts  (for  example  see  Neuts 
[1979])  has  been  left  untouched.  The  study  of  closed  networks  has  been  nearly 
Ignored  (e.g.  Gordon  and  Newell  [1967]).  The  interesting  network  decomposition 
ideas  of  Courtols  [1978]  deserves  attention  both  for  its  theory  as  well  as  its 
application  potential.  The  concepts  of  approximations  Including  diffusion 
approximations  and  heavy  traffic  approximation  have  not  even  been  mentioned. 

One  might  consult  Harrison  [1978]  to  start  into  this  area.  It  would  seem  that 
there  is  no  end  to  such  an  enumeration.  The  study  of  queueing  networks  is  an 
enormously  large  and  diverse  field.  Our  tutorial  has  at  best  "hit  the  high 
spots" . 
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